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Abstract The considered model will be formulated as related to "the fishing prob- 
lem" even if the other applications of it are much more obvious. The angler goes 
fishing. He uses various techniques and he has at most two fishing rods. He buys a 
fishing ticket for a fixed time. The fishes are caught with the use of different methods 
according to the renewal processes. The fishes' value and the inter arrival times are 
given by the sequences of independent, identically distributed (i.i.d.) random vari- 
ables with the known distribution functions. It forms the marked renewal-reward 
process. The angler's measure of satisfaction is given by the difference between the 
utility function, depending on the value of the fishes caught, and the cost function 
connected with the time of fishing. In this way, the angler's relative opinion about the 
methods of fishing is modelled. The angler's aim is to have as much satisfaction as 
possible and additionally he has to leave the lake before a fixed moment. Therefore 
his goal is to find two optimal stopping times in order to maximize his satisfaction. 
At the first moment, he changes the technique of fishing e.g. by excluding one rod 
and intensifying on the rest. Next, he decides when he should stop the expedition. 
These stopping times have to be shorter than the fixed time of fishing. The dynamic 
programming methods have been used to find these two optimal stopping times and 
to specify the expected satisfaction of the angler at these times. 
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1 Introduction 

Before we start the analysis of the double optimal stopping problem (cf. idea of 
multiple stopping for stochastic sequences in Haggstrom |[8]|, Nikolaev ifTSl ) for 
the marked renewal process related to the angler behavior, let us present the so 
called "fishing problem". One of the first authors who considered the basic version 
of this problem was Starr |fT9l and further generalizations were done by Starr and 
Woodroofe EB, Starr, Wardrop and Woodroofe EqI, Kramer, Starr IHl et al. The 
detailed review of the papers related to the "fishing problem" was presented by Fer- 
guson Q. The simple formulation of the fishing problem, where the angler changes 
the fishing place or technique before leaving the fishing place, has been done by 
Karpowicz II12I . We extend the problem to a more advanced model by taking into 
account the various techniques of fishing used the same time ( the parallel renewal- 
reward processes or the multivariate renewal-reward process). It is motivated by 
the natural, more precise models of the known, real applications of the fishing prob- 
lem. The typical process of software testing consists of checking subroutines. At the 
beginning many kinds of bugs are being searched. The consecutive stopping times 
are moments when the expert stops general testing of modules and starts checking 
the most important, dangerous type of error Similarly, in proofreading, it is natural 
to look for typographic and grammar errors at the same time. Next, we are looking 
for language mistakes. 

As various works are done by different groups of experts, it is natural that we 
would compete with each other. If in the first period work is meant for one group 
and the second period needs other experts, then they can be players of a game be- 
tween them. In this case the proposed solution is to find the Nash equilibrium where 
strategies of players are the stopping times. 

The applied techniques of modeling and finding the optimal solution are similar 
to those used in the formulation and solution of the optimal stopping problem for the 
risk process. Both models are based on the methodology explicated by Boshuizen 
and Gouweleeuw HI. The background mathematics for further reading are mono- 
graphs by Bremaud 13], Davis H) and Shiryaev ifTSl . The optimal stopping problems 
for the risk process are subject of consideration in papers by Jensen ifTol . Ferenstein 
and Sierocinski |6l|, Muciek |fT5|. A similar problem for the risk process having dis- 
ruption {i.e. when the probability structure of the considered process is changed at 
one moment Q) has been analyzed by Ferenstein and Pasternak-Winiarski Q. The 
model of the last paper brings to mind the change of fishing methods considered 
here, however it should be made by a decision maker, not the type of the environ- 
ment. 

The following two sections usher details of the model. It is proper to emphasize 
that the sUght modification of the background assumption by adopting multivariate 
tools (two rods) and the possible control of their numbers in use extort a different 
structure of the base model (the underlining process, sets of strategies - admissible 
filtrations and stopping times). This modified structure allows the introduction of 
a new kind of knowledge selection which consequently leads to a game model of 
the anglers' expedition problem in the section 11.21 and 12.21 After a quite general 
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formulation a version of the problem for a detailed solution will be chosen. However, 
the solution is presented as the scalable procedure dependent on parameters which 
depends on various circumstances. It is not difficult to adopt a solution to wide range 
of natural cases. 



1.1 Single Angler's expedition 

The angler goes fishing. He buys a fishing ticket for a fixed time fo which gives him 
the right to use at most two rods. The total cost of fishing depends on real time of 
each equipment usage and the number of rods used simultaneously. He starts fishing 
with two rods up to the moment s. The effect on each rod can be modelled by the 
renewal processes {Ni{t),t > 0}, where Ni{t) is the number of fishes caught on the 
rod /, / G 21 : = {1,2} during the time t. Let us combine them together to the marked 
renewal process. The usage of the i-th rod by the time t generates cost c, : [0,fo] 
3i (when the rod is used simultaneously with other rods it will be denoted by the 
index dependent on the set of rods, e.g. a, cf) and the reward represented by i.i.d. 
random variables xj'\x2'\ . . . (the value of the fishes caught on the i-th rod) with 
cumulative distribution function i/fl. The streams of two kinds of fishes are mutually 
independent and they are independent of the sequence of random moments when the 
fishes have been caught. The 2-vector process A^(f) = {Ni{t),N2{t)), f > 0, can be 
represented also by a sequence of random variables T„ taking values in [0,°°] such 
that 

7b =0, 

for n £ N, and a sequence of 2l-valued random variables i„ for « G N U {0} (see 
Bremaud |[3| Ch. II, Jacobsen ||9l). The random variable T„ denotes the moment of 
catching the n-th fish {Tq = 0) of any kind and the random variable 3„ indicates to 
which kind the «-th fish belongs. The processes Ni{t) can be defined by the sequence 

{{Tn,ln)}n=Q ^S: 

m)^i\T.,<,}\.,=i}- (2) 

Both the 2-variate process A^(f) and the double sequence {(7;,,3„)}"^q are called 
2-variate renewal process. The optimal stopping problems for the compound risk 
process based on 2-variate renewal process was considered by Szajowski ll22l . 
Let us define, for / G 21 and ^ G N, the sequence 

nll^ = \nf{n > n^ ' : i„ = i} 



' The following convention is used in all the paper: It^ = (xi ,X2, . . . ,Xj) for the ordered collection 
of the elements 
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and put Tjj^ = T {,} . Let us define random variables si — T,, ^ — T^^^ and as- 
sume that they are i.i.d. with continuous, cumulative distribution function Fi{t) = 
F{si'^ < t) and the conditional distribution function F^{t) = P(5/,'^ < f l^i'^ > 5). In 
the section lTTI the alternative representation of the 2-variate renewal process will be 
proposed. There is also mild extension of the model in which the stream of events 
after some moment changes to another stream of events. 

Remark 1. In various procedures it is needed to localize the events in a group of the 
renewal processes. Let £ be the set of indices related to such a group. The sequence 
{"f }r=o ^^^^ ^^^^ "0 ^ ^' "f+i iiif{« > «f : 3n G has an obvious meaning. 
Analogously, n'^(f ) := inf{n : r„ > f,3„ G £}. 

Let /,y e 21. The angler's satisfaction measure {the net reward) at the period 
a from the rod / is the difference between the utility function gf : [0,°°)^ x 21 x 
— > [0,Gf] which can be interpreted as the reward from the /-th rod when the 
last success was on rod j and, additionally, it is dependent on the value of the fishes 
caught, the moment of results' evaluation, and the cost function cf : [0, to] — s- [0, C"] 
reflecting the cost of duration of the angler's expedition. We assume that gf and cf 
are continuous and bounded, additionally cf are differentiable. Each fishing method 
evaluation is based on different utility functions and cost functions. In this way, the 
angler's relative opinion about them is modelled. 

The angler can change his method of fishing at the moment s and decide to use 
only one rod. It could be one of the rods used up to the moment s or the other one. 
Event though the rod used after s is the one chosen from the ones used before s its 
effectiveness could be different before and after s. Following these arguments, the 
mathematical model of catching fishes, and their value after s, could (and in practice 
should) be different from those for the rods used before s. The reason for reduction 
of the number of rods could be their better effectiveness. The value of the fishes 
which have been caught up to time f, if the change of the fishing technology took 
place at the time s, is given by 

!G21 n=l n=l n=l 

where = I^fj^xi'^ and M, = I^j^iM^ We denote aI, = (m/'^M^). Let 
Z{s,t) denote the angler's pay-off for stopping at time f (the end of the expedition) 
if the change of the fishing method took place at time s. If the effect of extending 
flie expedition after s is described by : 9^+^ x 21 x [0,fo] x x [0,fo] [0,Gy], 
j G *8, minus the additional cost of time c^(-)' where : [0,ro] [0,Cj'] (when 
card(*8) = 1 then index j will be abandoned, also c^ = L>g<b used, which 

will be adequate). The payoff can be expressed as; 
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Z(.,r) = <' (4) 
+g''iM,,iN{s),s,M-^,t)-c^{t-s) if 5 < f < fo, 

-C iffo<f- 

where the function c°(f), g°(77l,/,f) and the constant C can be taken as follows: 

c"(0 = liicr(f), = lLi§r(^r,;,0. C = Cf + C2° + C^ After mo- 

ment s the modelling process is the renewal-reward one with the stream of i.i.d. 
random variables X„ ' at the moments Tn ' (i.e. appearing according to the renewal 
process A^3(f)). With the notation w''(7?t,;,i,m,f) = w"'{Tit,i,s) + (Tit ,i,s,fh,t) — 
c^{t — s) and w°(»t,/,f) = g^Cntjij) — c°(f), formula Q is reduced to: 

z(5,f) =zW)}(,,f)i^,<^.<,^j +z{3N(-')>(.,f)i{.<,}, 

where 

Z-t'> (i , f ) = I{r<.<ro} V^"" (^r , / , f ) + hs<t<to}w'' s ,i,s,Mt,t)~ I{ro<r}C. 



i.2 The competitive fishing 

When the methods of fishing are operating by separated anglers then the stopping 
random field can be built based on the structure of the marked renewal-reward pro- 
cess as a model of the competitive expedition results. One possible definition of 
pay-off is based on the assumption that each player has his own account related to 
the exploration of the fishery. The states of the accounts depend on who forces the 
first stop for changing technique, under which circumstances and what techniques 
they choose. The first stopping moment, the minimum of stopping moments cho- 
sen by the players, is after the moment of the event (catching fish) T„ by the rod 
3„ and the reward functions depend on the type of fishing which gives recent fish 
(i.e. where j = 3,,). The player's pay-off wf{ntj,t) = gf{ntj,t)-cf{t) . The 
part of the pay-off which depends on the second chosen moment, which stops the 
expedition, is different for the player who forces the change of fishing methods (the 
leader) by himself and the other the opponent. The leader is the responsible angler 
for determining the expedition deadline. 

Lets assume for a while that the i-th player, / = 1,2, will take the rod of the 
opponent and gives his rod to him. It is not a crucial assumption anyway and the 
method of fishing after the change can be different from both available before the 
considered moment. The method of treatment of the case without this assumption 
will be explained later (see page|9|l, when the behavior of the player in the second 
part of the expedition will be formulated. Define the function 

'^H'^J^s^k,m,t)^wf{nlj,s)+g°{ntj,s,k,fh,t)~c^{t-s) 
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for y G 21, A: G S, where j is the rod by which the fish had been caught just before the 
moment of the first stop and k is the technique used by ;-th player after the change 
(the denotation —k is used for a compUmentary rod or player who has decided, 
which is appropriate). It describes the case when the player deciding to change the 
method chooses the perspective technique of fishing as the first one. Presumably he 
will explore the best methods with improvements and the second angler will use the 
rod which is not used by the leader The pay-off of the players, when i-th is the one 
who forces the first stop, has the following form: 

Z^i{J,s, t) = I{r<,<,o}|° ,(^( , 7, f ) + I{s<r<ro}*^ (^.v, S, i,M', , f ) - 1[,^<,}C. (6) 

In the above pay-offs it is assumed that the final stop can be declared at any moment. 
The change of techniques declaration each player makes just after an event at his rod 
(the catching fish at his rod) as long as on the opponent's rod there is no event. The 
details of the strategy sets and the solution concept are formulated in the further 
parts of the paper. 

The extension considered here is motivated by the natural, more precise mod- 
els of the known real applications of the fishing problem. The typical process of 
software testing consists of checking subroutines. Various types of bugs can be dis- 
covered. Each problem with subroutines generates the cost of a bug removal and 
increases the value of the software. It depends on the types of the bug found. The 
preliminary testing requires various types of experts. The stable version of subrou- 
tines can be kept by less educated computer scientists. The consecutive stopping 
times are moments when the expert of the defined class stops testing one module 
and the another tester starts checking. Similarly as in the proof reading. 



2 The optimization problem and a two person game 
2.1 nitrations and Markov moments 

Let the sequences of pairs 3n)}jr=o 2-variate renewal process (2l-marked 
renewal process) defined on {£2 , P). According to the denotation of the previous 

section there are three renewal processes {7;/'^ }jr=0' ' ^ 1:2,3, and denoted by T„ = 

rJ^"L There are also three renewal-rewarded processes {{T,j'\Xn'^)}°°r^, i = 

1,2,3 . By convention let us denote X„ = xjf"}^ >. The following CJ-field generated 
by history of the 2l-marked renewal processes are defined 

■^t = -^t^ = c (^0 , 7b , 30 • • • , Xfj{t) , 7}v(r) , iN{t) ) , (7) 
for f > 0. This c7-field can be defined as 
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= cj{(A^(^),X^(.),3;v(.)),0 < ^ < f, / e 2t}. 



Definition 1. Let 5^ be a set of stopping times with respect to c-fields {^t\, f > 0, 
defined by O. The restricted sets of stopping times are 

■%.K^{^^sr : x> 0, r„ < T < Tk) (8) 

for n G N, « < A' are subsets of 3" . The elements of S'n.K are denoted x„ k_ ■ 

The stopping times T e ^ have nice representation which will be helpful in the solu- 
tion of the optimal stopping problems for the renewal processes (see Bremaud |3j). 
The crucial role in our subsequent considerations plays such a representation. The 
following lemma is for the unrestricted stopping times. 

Lemma I. If T G ^ then there exist R„ £ Mes{^„) such that the condition tA 
Tn+i = (Tn +Rn)/\T„+i on {t > Tn} ci.s. is fulfilled. 

Various restrictions in the class of admissible stopping times will change this repre- 
sentation. Some examples of subclasses of ^ are formulated here (see Lemma[T]). 
Only a few of them are used in optimization problems investigated in the paper (see 
pagelH Corollary[T])- 

Let = (T(^,2t^;^J3}^j,j3}^^^^^^CT^^^^^^^y,CT^^^^^P be the a-field gener- 

ated by all events up to time t if the switch at time s from 2-variate renewal 
process to another renewal process took place. For simplicity of notation we se0 

^i'^ := ^,,} , -^n ^T,„ ■■= ^{3}- Let Mes{^„) (Mei(^i'^)) denote the 

set of non-negative and ^„ (^i'^)-measurable random variables. From now on, 
3" and l^'^ stands for the sets of stopping times with respect to cr-fields and 
{^s.r,0 < i < f }, respectively. Furthermore, we can define for n G N and n < K the 
sets 

L = {Te3:T>0, r« < T < Tk}; 

3. #„|f'""^ = {Te^:T>o, ^«<T<^^,v,T^[^^^^,2[7v^» ,]} 

where Stt-} a\ {/}, T,^- min^ 

4. = {T e ^ : T > T,P ,v,T i [r;^"\r,2t7 vr« * , ]}; 

5. .^If, ^{xe.3':0<s<T, T,P <T< Tk}. 

The stopping times T G and T G ^^'^ can also be represented in the way shown 
in Lemma [U 



^ For the optimization problem there are two epochs: before the first stop, where there are some 
pay-offs, the model of stream of events, and after the first stop, when there ai'e other pay-offs and 
different streams of events. In section|3]this will be emphasized, by adopting adequate denotations. 
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Leiiuna 2. Let the index i be chosen and fixed. 

1. For every T € J^^'J and « £ N there exist rI'"' £ Mes{^n'^ ) such that T A rj^\ = 

(rj'^ A 7;|_|\ on {t^'J > rj'^} a.^. is fulfilled. 

2.1fze fl«^f « 6 N f/iere ex/if 7?/,'^ G Mes{.^n'^) such that the condition X A 

Tn2\ = {Tn'^+Rn^) A on {x > ri'^} is fulfilled. 

Obviously the angler wants to have as much satisfaction as possible and he has 
to leave the lake before the fixed moment. Therefore, his goal is to find two optimal 
stopping times t" and so that the expected gain is maximized 



EZ(t°*,t''*) = sup sup EZ(t",t''), 



(9) 



where T° corresponds to the moment, when he eventually should change the two 
rods to the more effective one and x^ , when he should stop fishing. These stop- 
ping moments should appear before the fixed time of fishing fo. The process Z{s,t) 
is piecewise-deterministic and belongs to the class of semi-Markov processes. The 
optimal stopping of similar semi-Markov processes was studied by Boshuizen and 
Gouweleeuw (l] and the multivariate point process by Boshuizen [2J- Here the 
structure of multivariate processes is discovered and their importance for the model 
is shown. We use the dynamic programming methods to find these two optimal 
stopping times and to specify the expected satisfaction of the angler. The way of the 
solution is similar to the methods used by Karpowicz and Szajowski ifTSll . Karpow- 
icz lfT2l and Szajowski ll22l . Let us first observe that by the properties of conditional 
expectation we have 



EZ(t"*,t^*) = sup E{E 



Z(t°,t''*)|^,c. 



} = sup E/(t"), 



where 



J{s)^E Z{s,x^')\^s 



esssupE 



z{syM. 



(10) 



Therefore, in order to find T° and x'^ , we have to calculate J{s) first. The process 
J{s) corresponds to the value of the revenue function in one stopping problem if the 
observation starts at the moment s. 



2.2 Anglers' games 

Based on the consideration of the section 11.21 a version of competitive fishing is 
formulated here. There are two anglers, each using one method of fishing at the 
beginning of an expedition and an additional fishing period after a certain moment 
by another method up to the moment chosen by a certain rule. The random field 
which is the model of payoffs in such a case is given by (|5]l and (|6]l. The final 
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segment starts at the moment when one of the anglers wants it. Let t, G i ~ 21, 
be the strategies of the players to stop individual fishing period and switch to the 
time segment which is stopped at moment a determined by one angler (let us call 
them a leader). The payoffs of the players are 

V/,-(Ti,T2) = Zi{iM{Ti/\t2)>'^l^'^2,<y^'^^^)l{z,^z,} (11) 

The aim is to find a pair (rj^, T2 ) of stopping times such that for / e { 1 , 2} we have 

Ev/,«,<-)>Ev/,-(t;,t*,.). (12) 

The optimization problem of the angler and the game between two anglers will 
involve the construction of the optimal second stopping moment. 



3 Construction of the optimal second stopping time 

In this section, we will find the solution of one stopping problem defined by (fTOl i. 
We will first solve the problem for the fixed number of fishes caught, next we will 
consider the case with the infinite stream of fishes caught. In this section we fix 
s - the moment when the change took place and m = Ms - the mass of the fishes 
at the time s. Taking into account various models of fishing after the first stop it 
is needed to admit various models of stream of events. Assume that the moments 
of successive fishes catching after the first stop are T„ and the times between 
the events are i.i.d. with continuous, cumulative distribution function F(f ) with the 
density function f(f )and the fishes value represented by i.i.d. random variables with 
distribution function H(f ) (for conveniences this part of expedition is modelled by 

the renewal process denoted (T^f ^X,P^)). 



3.1 Fixed number of fishes caught 

In this subsection we are looking for optimal stopping time Tq ^ := 

Z(s, Tj^*)|,^J =esssupE Z(i,T|)|^,j , 



E 



(13) 



where i > is a fixed time when the position was changed and K is the maximum 
number of fishes which can be caught. Let us define 



r„V= ess sup E Z(^,T„V)|^^ 



(z ITS 



-E 



« = /:,...,!, (14) 
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and observe that = Z(i, )■ In the subsequent considerations we will use the 
representation of stopping time formulated in Lemma [T] and |2] The exact form of 
the stopping strategies are given in the following corollary. 



Corollary 1. Let i G 21. //t° e ,5^^'}, e .3^\ then there exist e Mes{^r ) and 

{'■} 

n+l 



e Mes{,^^) respectively, such that for conditions T° A rjj^ = {T,P + /?°) A 7;''_J^ 



on {t° > ri'^ } a.s. and A tJH = (xj^^ +R°)A rj|| on {t° > A 7;P* } a.i. are 
valid. 

Now we can derive the dynamic programming equations satisfied by fj',*^. To 
simpUfy the notation we can write Mt = Mf for f < 5, M,!'^ = Mj^^i , = ■'^^,{3} and 
Fj = 1 — F/. The payoff functions are simplified here to |°(ot) =g°(7wi,OT2,/,f)I{„,j+„,2=„,}(mi,OT2), 

=g''(mi,m2,/,i,OT,f)I{,7i-mi-m2=m} 

Lemma 3. Lef s >0 be the moment of changing fishery. For n = K — 2,...,0 

ri.-zis,TPi ^^^^ 

= eSSSUp„bgM^,(,^,;) ^nAM^^sMnJn ,K) ^.S., 



where 



A,K {m, 5, OT, f , r) = I{,<,^} I F{r) [l{r<to-t} (m, s,m,t + r)- CI{,->,q_,}] 



a«c/ f/iere e.i/ifi 7?,^* e Mes{^^) such that 



CI 



{t>to} 



r^K = ^?„.Jf(M,,^,M^,7;/^^/^f ) (16) 



X ^ rj^^ andw^ {m,s,m,t) ~ w'^{m,s) +g'^{m — m) — c''(f — i) where w'^{m,t) = 
r(m)-c°(r). 

i?emar/t 2. Let {^fl^Lj, Z?^* = 0, be a sequence of ^,^-measurable random vari- 
ables, « = 1 , 2, and tj;;';^ = Ainf{/ > n : Rf* < sjH}. Then r„;^ = E \z{s, T^'^)\.^^ 
fovn<K-l, where ri'V = T„*s +rK, . 

Proof of remark. |2] It is a consequence of an optimal choice R'^* in ( fTsT l. 
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Proof of lemma. |3]First observe that the form of the for the case rj^^ > to 
is obvious from (|4]i and (fT4l i. Let us assume ( flST l and (fTST i for « + 1,« + 2, . . . For 
any T e =^„^^ (i.e. T > 7;P> we have {t < T^^j} = {TATjl\ < t}1\} = {T,P < 
T^ll}. It impUes 



(18) 



Suppose that T^^J^ < to and take any ^ G /f- According to (fTSl l and the 
properties of conditional expectation 



E[Z(.,T)|^:] =E 
+ E 



{3} 



E 



I^^{3^,^^_^jE[Z(.,TVr,{_f|)|^,;+i|^„^ 



Let a e - For every T G we have 



^{3} 



We have 



E[Z(,,T)|^^1=E 



I{5{3)<„,}E[Z(^,(T)|^+i]|^„ 



< sup {E 



{3}_ 



It follows sup^g,3..E[Z(i, t)|^^] < E[Z(5, t*k\-^;,] < sup^g^^^b E[Z(i,T)|,^^'] where 
the last inequality is because T*i^ G We apply the induction hypothesis, which 
completes the proof. 

Lemma 4. fj'/^ = YK^n^^rnT^n^^) f^'' n = K , . . . ,0, where the sequence of functions 
Yj'"' is given recursively as follows: 



12 A. Karpowicz, K. Szajowski 



Yo i'n,t) = l{,<to}W {m,s,m,t)-Cl{,>,g}, 

r>0 



Y]''"im,t) = sup K-yW,, {m,s,m,t,r)-Cl^t>to}, (19) 



where 



Kl{m,s,m,t,r) ^ F{r)[l{,<,^_,}W^ {m,s,m,t + r) -Cli,->ro-t}] 
dF{z) f 5{m+x,t + z)dH{x). 



Proof of lemma. |4] Since the case for f > fo is obvious let us assume that 
Tn^^ < fo for n G {0, ...,K —I}. Let us notice that according to Lemma|3]we obtain 

K ~ To^^ {^K''^P^)' ^^^^ proposition is satisfied for ii = K. Let n = K — \ 
then Lemma[3]and the induction hypothesis leads to 

r^_i,^= esssup |F(/?Li)[Ir^i, >,}w\M,,s,M'k_„4%+rI_,) 



where M^_i +Xp\ Tp^ = Tp\ + and the random variables 

and S^^^ are independent of ,^s.K-i- Moreover M^k-i ^'^'^ ^i./f-i- 
measurable. It follows 

r^_i.^= esssup (f(4_i {3} w''(M,,.,Mi_i,rj!\+4_i) 



t^K-l>'o-^K-ll Jo Jo 
= 7j'^'(M^_i,ri!>)«... 

Let « e { 1 1 } and suppose that i;;^^ = ■y^'^^ (M;; , T,P). Similarly Uke before, 
we conclude by Lemma[3]and induction hypothesis that 



C-ix= esssup (F(/?tJ[I ^,.w\M,,sM;,-uT,j^\+Rli) 



thereforei;;:i ^ = 7^^^„_^)(M^i,7;|_^{). 
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From now on we will use a, to denote the hazard rate of the distribution Ft (i.e. 
ai = fi/Fi) and to shorten notation v/e sel A' (a) = 'E [g (a + X^'^) — g {a)], where • 
can be a or b. 

Remark 3. The sequence of functions y^'"' can be expressed as: 

fo"\m,t) = I{,<,(,}vv''(m,i,m,f) -CI{(>,^}, 



f/"\m,t) = I{r<ro}|i^' {m,s,m,t) +y j {m- m,t - sJq - 1) j - Cl^i^to} 
and {a,b, c) is given recursively as follows 
y^{a,b,c) = 

y'j{a,b,c) = max {a,b,c,r), 

■' 0<r<c 

where (l)l{a,b,c,r) = J^F{z){a2iz) [a'' (a) +E5{a +X^^\b + z,c - z)] - c''' { 
z)}dz. 

Proof of remark. [3]Clearly 



where has c.d.f. F andX^^ has c.d.f. H. Since F is continuous and fc^,,, (jn,s,m,t,r) 

0-1 

is bounded and continuous for t G \ {to}, the supremum in ST9[ can be changed 
into maximum. Let r > fo — f then 

ff^..Ki,m,f,r) =E[l^5{3)<,„_,l^l"i(m + X«,r + 5«)] -CF{to-t) 



< E 



= K^.,„ {m,s,m,t,to-t). 
0-1 

The above calculations cause that Yj''"(m,t) — I|,<,jj}maxo<r<ro-( (pj{m,s,m,t,r) — 

CI{r>ro}, where (p^ {rn,s,m,t ,r) ^ F(r)vv'' (to, i, m, f + r) -f-E I|j{3} y*:"j {tn + X^^\t + S''^^) 

Obviously for S^^^ <r and r < fo — f we have 5^^^ < fo therefore we can consider the 
cases f < fo and f > f o separately. Let f < fo then ^{^'"'(mjf) = {m^s^m^t) and the 
hypothesis is true for j = 0. The task is now to calculate '/j'"\im,t) given 7*'"' (•,•)■ 
The induction hypothesis implies that for f < f o 
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(Pj^\{m,s,m,t,r) = F{r)w^ {m,s,m,t + r) +¥, 

= |°(m)-c°(i)+F(r) \g''{m-m)-c''{t-s + r) 

+ rf{z){Eg''{m-m+X^^^)-c\t-s + z) 
Jo 

+ Ey'^ {in-m+X^^\t-s + z,to-t-z)}dz. 
It is clear that for any a and b 

F{r)[g\a)~c\b + r)']=g\a)~c\b) 

- /'{fW \8\a)-c\b + z)] +F{z)c'''{b + z)}dz, 
Jo L J 

therefore 

(Pj+l{m,s,rh,t,r) = w'^{m,s,fh,t) + J F(z){a2(z)[4''(m — m) 

+ Ey^j{m-m+X^^\t-s + z,tQ-t-z)]-c'''{t-s + z)}dz, 
which proves the theorem. The case for t > tois trivial. 



Following the methods of Ferenstein and Sierocinski ||6l, we find the second op- 
timal stopping time. Let B = *8([0,oo) x [0,fo] x [0,fo]) be the space of all bounded, 
continuous functions with the norm ||5|| = sup^^ ^.\8{a,b,c)\. It is easy to check 
that B with the norm supremum is complete space. The operator ^>'' : B — B is 
defined by 

{<p''5){a,b,c) = max (l)^{a,b,c,r). (20) 



0<)-<t 



Let us observe that y^(a,^7,c) = {<P^y^_^){a,b,c). Remark[3]now impHes that there 
exists a function r^*{a,b,c) such that3'^(a,fe,c) = {a,b,c,r^*{a,b,c)) and this 

v._i 

gives 



f/'^im,!) = I{,<,(,}|w {m,s,m,t) 

+ ^yf ^{m-m,t-s,to-t,rf (m - m,f - i,fo - f ))| - CI{,>,q}. 

The consequence of the foregoing considerations is the theorem, which determines 
optimal stopping times ^ in the following manner: 



Theorem 1. Let Rf^ = r^'-iiM- - M,, 7;.^^^ -s,to- T^"^)for ; = 0, 1 , . . . 



over f]nK^^ ^ inf{/ > n : Rf < Sj^\ }, then the stopping time T,, ^ = 



v 



%.K %.K 



is optimal in the class and F^j^ = E Z{s^ . 
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3.2 Infinite number of fishes caught 

The task is now to find the function J{s) and stopping time t'' , which is optimal in 
class ^\ In order to get the solution of one stopping problem for infinite number 
of fishes caught it is necessary to put the restriction F(fo) < 1- 

Lemma 5. IfF{to) < 1 then the operator B ^ B defined by ( |2Ql ) is a contrac- 
tion. 

Proof of lemma. |5] Let 5; G B assuming that / e {1,2}. There exists p,- such that 
[^^8i)(a,b,c) = (j)^,{a,b,c,pi). We thus get 



i<P''5i){a,b,c)-{cP^52){a,b,c) ^ 0l{a,b,c,pi)^<Pl{a,b,c,p2) 

< (l)lia,b,c,pi)-^l{a,b,c,pi) 

= r dF{z) [ [8i-5,]{a+x,b + z,c-s)dn{x) 
Jo Jo 

< ! ' d¥{z) I sup|[5i-52](fl,/7,c)|t/H(x) 

JO Jo aM.c 

< F(c) - 52|| < F(ro) - < C - SzH , 

where < C < 1. Similarly, like as before, {<i>^ &2){a,b,c) - {<i>^ 5\){a,b,c) < 
C ||52 - Finally we conclude that ||c?>''5i - c?>''52|| < C ||5i - 52|| which com- 
pletes the proof. 

Applying Remark[3] Lemma|5]and the fixed point theorem we conclude 

Remark4. There exists y'' e B such that y*" =^^y^ and lim/f^oo Hyl -y'' || =0. 

According to the above remark, is the uniform limit of y^, when K tends to 
infinity, which implies thaty'' is measurable and 7* '" = limAr^oc Y'k' is given by 

f'"'(m,f) = I{,<ro} w^{m,s,m,t)+y^{m~m,t~s,tQ-t) -C\{,>,^y (21) 

We can now calculate the optimal strategy and the expected gain after changing the 
place. 

Theorem 2. IfF{to) < 1 and has the density function f, then 

(ijfor n G N the limit = lim^r^oo T„''^ a.s. exists and < to is an optimal 

stopping rule in the set S/'^ H {t > T,/^^}, 
(ii)E {Z{s,xt)\.^f\ = f ''"(M^,ri'}) a.s. 

Proof, (i) Let us first prove the existence of T,f . By definition of r^,\^^ we have 
r,'K+i = esssup E[Z(i,T)|,^'] =esssupE[Z(i,T)|^;] V esssup E [Z(i, t)|^^] 



E 
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thus we observe that T,f^_|_i is equal to T,fjj. or a*, where T^^ G 5^*'^ and C7* e 
■S^XK+i respectively. It follows that T,f]5._|_i > t^J'a: which implies that the sequence 
'^n*K nondecreasing with respect to K. Moreover Rf* < to — T-^^^ for all / e 
{0,...,K} thus TjJ'Jj. < fo and therefore T^* < to exists. 

Let us now look at the process ^"{t) = (f,M/,y(f)), where s is fixed and V{t) = 
/ — T^^j^y ^"(t) is Markov process with the state space [s,to] x [m,oo) x [0,°°). In a 
general case the infinitesimal operator for is given by 

V-"'(f,'H,v) = ^/-(f,,^i,v) + i-p-^-.'«(f,,7j,v) 



«2(v)| / /.""(f,x,OMH(x)-/''«(f,m,v) 



where p''-"'{t,m,v) : [0,°°) x [0,°°) x [0,oo) — > K is continuous, bounded, measurable 
with bounded left-hand derivatives with respect to t and v (see fl] and ifTTl ). Let 
us notice that for t >s the process Z{s,t) can be expressed as Z{s,t) — •'"{£,■'' (t)), 
where 



l''(M,) -c'^G?) +I''(m; -m,) -c''(f -i) if i < f < fo, 

-C iffo<f. 



It follows easily that in our case Ap'''"'{t,m,v) = for to <t and 

Ap''^'"{t,m, v) = a2(v) [El'' (m - m) - |''(ot - m)] - c^t{t - s) (22) 

for i < f < fo- The process p'-"'{t,'{t)) - p'-"'{B,'{s)) - ^ {Ap''"'){B,'{z))dz is a mar- 
tingale with respect to < f) which is the same as ,^5,. This can be found 
in H. Since < fo, applying the Dynkin's formula we obtain 



E 



a.s. 
(23) 



From 



we conclude that 



'■n.K , 



.6* 



{3} 



Moreover let us check that 
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{3} 



c''(Ti';-.)-c''(7;PJ-.) 



E|''(M^+xW-m)-|''(M;;-m) 



where the two last inequalities result from the fact that the functions g'^ and c'' are 
bounded. On account of the above observation we can use the dominated conver- 
gence theorem and 



Hm E 



{3) 



{Apni^-\z))dz\.J^;; 



(24) 



Since < to applying the Dynkin's formula to the left side of ( l24b we conclude 
that 



E 



{3} 



{Ai/n{^^{z))dz\^^ 



:E 



Combining ( |23] ). ( l24b and dZSl ) we can see that 



(25) 



hm E 



(26) 



hence that lim^r^oo E 



Z(i, T^f j^) I = E [Z{s, T° ' ) I ,^,^\ . We next prove the opti 



mality of t^* in the class .T' n {t,^ > T,/"'^}. Let T £ 5^' n {T,f > T,P^ and it is 
clear that t A 7^^^ G As T^, is optimal in the class ^„^^ we have 



lim E 



> lim E 



p'''"{^'{tatP))\^:; 



(27) 



From ^ and (|27li we conclude that E [//'"'(^"(rf > E [//•'"(^■^'(t))|^*] 

for any stopping time T G =5^'' n {t > rj^*^ }, which implies that xj^* is optimal in this 
class. 

(ii) Lemmagland daSll lead to E Z{s,T^*)\.^,'; ^ f^' {M^^j!^^''). 

□ 

The remainder of this section will be devoted to the proof of the left-hand differ- 
entiabiUty of the function Y'"\ni,s) with respect to s. This property is necessary to 
construct the first optimal stopping time. First, let us briefly denote 5(0,0,c) G B by 
5(c). 

Lemma6. Lef v(c) = 4>^5{c), 5{c) G B and |5|(c)| < Ai for c G [0,fo) then 

|v;(c)| <A2. 
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Proof of lemma. |6] First observe that the derivative v:^(c) exists because 
v(c) = maxo<,<£ ''(c,r), where ^^(c,r) is differentiable with respect to c and r. 
Fix /z_e (0,fo -c) and define 5\ (c) = 5{c + h) e B and 52(c) = 5(c) e B. Obviously, 
||0''5i-0''52|| > l^''^! {c)~<P^52ic)\ ^ |0''5(c + /!)-^^5(c)| and on the other 
side using Taylor's formula for right-hand derivatives we obtain 

||5i - 52II = sup|5(c + /i) - 5(c)| < /2sup|5+(c)| + \o{h)\. 

c c 

From the above and Remark[8]it follows that 

-c{sup|a;(c)|+M!^}< ^(-+^)-^(-^ <c{sup|a;(c)| + m| 

and letting h 0+ gives |v|(c)| < CAi = A2. 

The significance of Lemma |6] is such that the function y (fo — s) has bounded left- 
hand derivative with respect to s for s g (0,fo]. The important consequence of this 
fact is the following 

Remarks. The function 7'''" can be expressed as 7'''"(m,s) = I{i<,Q}M(m,s) — CI{s>,g}, 
where u{m,s) = g°'{m) — c''(i) +^''(0) — c''(0) +y^{tQ —s) is continuous, bounded, 
measurable with the bounded left-hand derivatives with respect to s. 

At the end of this section, we determine the conditional value function of the second 
optimal stopping problem. According to ( fTOl i. Theorem|2]and Remark|5]we have 



7(i)=E 2(5,1"*)!^, =/''"'(M„i) a.s. (28) 



4 Construction of the optimal first stopping time 

In this section, we formulate the solution of the double stopping problem. On the 
first epoch of the expedition the admissible strategies (stopping times) depend on 
the formulation of the problem. For the optimization problem the most natural are 
the stopping times from ,^ (see the relevant problem considered in Szajowski 1221 ). 
However, when the bilateral problem is considered the natural class of admissible 
strategies depends on who uses the strategy. It should be ,'7^'^ for the /-th player 
Here the optimization problem with restriction to the strategies from the at the 
first epoch is investigated. 

Let us first notice that the function u{m, s) has a similar properties to the function 
{m,s,fh,t) and the process J{s) has similar structure to the process Z{s,t). By 
this observation one can follow the calculations of Section [3] to get J{s). Let us 
define again I7,jf = esssup^ogfl?_^E[7(T")|^„] , n = K,. . . ,1,0, which fulfills the 
following representation 
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Leiiuna 7. r„ K = 7a:-«(Mi'^, T'n far n = /T, . . . ,0, where the sequence offanc- 
tions Yj can be expressed as: 

7o(OT,i) = I{,<,Q}M(m,i)-CI{,>,^}, 

Yj(m,s) ^ I{.,<,(,}|M(m,i)+3'°('M,i,fo-i)| -CI{,>,q} 

and y'j{a,b,c) is given recursively as fallows: 
y^{a,b,c)=0 

y'^{a,b,c) ~ max {a,b,c,r) 

■' 0<r<c - J- 1 

where 

(^l{a,b,c,r) = Fi{z){aiiz) [A°{a) +E8{a +x^^\b + z,c ~ z) 

- {y^'_{c~z)+c"'{b + zj)}dz. 

Lemma|7]corresponds to the combination of Lemma|4]and Remark[3]from Subsec- 
tion [TT] Let the operator 0'' : B B be defined by 

(4>°5)(fl,Z7,c) = max <j)^{a,b,c,r). (29) 

0<r<c 

Lemma|2]implies that there exists a function r^ ■{a,b,c) such that 



Yj [m, s) = I{,.<,g} |m(ot, s) + (to, s, to - s, rlj {m, s,to-s))j- CI{,>;q} . 
We can now state the analogue of Theorem[Tl 

Theorems. Let Rf = rf_,.(M,,7;.^'^fo - T,^^^) and T]n.K = K Mnf{i > n : Rf* < 
^j+l}' tl^^f^ '^nK = ^i|„'l +^n„ K optimal in the class ,'7n,K andF^ K = E y(T°^)|j^„ 

The following results may be proved in much the same way as in Section[3] 

Lemma 8. IfFi (fo) < 1 then the operator cf>o ; B — )- B defined by ^29\ is a contrac- 
tion. 

Remark 6. There exists e B such that y° = <i>°y^ and UmA:^c» -y"^!! = 0. 
The above remark implies that y = lim/f^oo Yk is given by 

Yim,s) ^ l{s<,a} [u{m,s) +y''(m,s,tQ-s)]-Cli,y,^y (30) 
We can now formulate our main results. 
Theorem 4. IfFi (fo) < 1 and has the density fanction f\, then 
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(i) /or « G N the limit ~ lim/(-_!.oo T^f^ a.s. exists and T° < fo is an optimal stop- 
ping rule in the set £/' r\{T > T^^^, 

(ii) E [/{xfMn] = y{M„jy^) a.s. 

Proof. The proof follows the same method as in Theorem |2l The difference lies 
in the form of the infinitesimal operator. Define the processes £,{s) = {s,Ms,V{s)) 
where V{s) = s~ T^^J^y Like before £, (s) is the Markov process with the state space 
[0,°o) X [0,00) X [0,00). Notice that J{s) = p{^{s)) and p{s,m,v) : [0,to] x [0,°°) x 
[0, °o) — ?• M continuous, bounded, measurable with the bounded left-hand derivatives 
with respect to s and v. It is easily seen that A/:'(i,m, v) = ai{v)[Eg° {tn + x^^^) — 

g°' (m)] — y°l_ (fo — i) + c'^' (s) for i < fo. The rest of the proof remains the same as 
in the proof of Theorem |2] 

□ 

Summarizing, the solution of a double stopping problem is given by 

EZ(t"*,t''*) = Ey(T"') = 7(Mo,rJ'^) = 7(0,0), 
where and T*" are defined according to Theorem|2]and Theorem|4]respectively. 



5 Examples 



The form of the solution results in the fact that it is difficult to calculate the solution 
in an analytic way. In this section we will present examples of the conditions for 
which the solution can be calculated exactly. 

Remark 7. If the process ^2(0 = A/:i* '"(^ '^(f )) has decreasing paths, then the second 



hi ,f0 



: A/7'''"(^-^ (f)) < 0} on 



optimal stopping time is given by = inf{f G 

the other side if ^2(0 has non-decreasing paths, then the second optimal stopping 
time is equal to fo. 

Similarly, if the process 1^1 (s) = A/?(^ (s)) has decreasing paths, then the first opti- 



ri^>,fo 



: A/?(^ (s)) < 0} on the other 



mal stopping time is given by T,f = inf{s 6 
side if ^1 (s) has non-decreasing paths, then the first optimal stopping time is equal 
tofo. 



Proof. From dB) we obtain E [Z{s, T^* ) | JT;] = Z{s, T,y^)+E 
a.s. and the application results of Jensen and Hsu ifTTll completes the proof. 



r'UAp'''"m'{z))dz 



□ 



Corollary 2. If S^^^ has exponential distribution with constant hazard rate a2, the 
function g'^ is increasing and concave, the cost function c^ is convex andt2^n = T,P\ 
mi = Mi then 
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=inf{fg [f2,„,fo]: "2 [E^''K+^^'^- ^) <c'''(f-^)}, (31) 

where s is the moment of changing the place. Moreover, if S^^^ has exponential 
distribution with constant hazard rate (X\, g'^ is increasing and concave, c° is convex 
andti ,, ~ m„ = M,{'^ then 

xf = miis g [fi,„,fo] : ai [e|"K -|"K)] < c"' {s)] 

Proof. The form of t"* and x^* is a consequence of Remark |7] Let us observe 
that by our assumptions 1^2 (0 = 01.2^^ {M^, — m) — c^' [t — s) has decreasing paths for 
t e [T,\^\t^^^). It suffices to prove that C2(ri^^) - l^iiT^-) = a2[4''(M^ -m) - 
A^{M'^_^-m)\ < Oforalln G N. 

It remains to check that y^l{tQ — s) ^Q>. We can see that T''* = T''*(5)is deterministic, 
which is clear from ( l3Tl i. Let us notice that if i < fo then combining (IZSl l. ( |26] | and 

m gives f ^'"(m,i) = E [Z(i,T''*)|^v] = Z(i,i) + E (A/7^''")(^-'(z))£/z|^. 
By Remark|5]it follows that 



/(fo-i)=E 
and this yields 



{Ap-"^){^\z))dz 



a2A\0)-c'^{z-s) 



dz 



^'\'i{z^s)dz + T^*'{s) \a2A\Q)-c'2{x^l{s)^s) 



(32) 



- a24^0)-4(0) 
= c'^{T^*{s)-s)-c'M - [a2A\Q)-c'M 
The rest of proof runs as before. 



□ 



Corollary 3. If for i — 1 and i = 2 the functions g, are increasing and convex, ci 
are concave and S^'^ have the exponential distribution with constant hazard rate ai 
then T°* = = to for n G N. 

Proof. It is also the straightforward consequence of Remark]?] It suffices to check 



— 



to- 



that y_ (to — s) is non-increasing with respect to s. First observe that t" {s 
Considering (|32] | it is obvious that y'^ {to ~ s) ~ a2A^ (0) — c'2 {to — s) and this com- 
pletes the proof. 

□ 



22 

6 Conclusions 
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This article presents the solution of the double stopping problem in the "fishing 
model" for the finite horizon. The analytical properties of the reward function in 
one stopping problem played the crucial rule in our considerations and allowed us 
to get the solution for the extended problem of a double stopping. Let us notice that 
by repeating considerations from Section |4] it is easy to generalize our model and 
the solution to the multiple stopping problem but the notation can be inconvenient. 
The construction of the equilibrium in the two person non-zero sum problem formu- 
lated in the section|2]can be reduced to the two double optimal stopping problems 
in the case when the payoff structure is given by (|5]l, (|6]l and ( fTTT i. The key assump- 
tions were related to the properties of the distribution functions. Assuming general 
distributions and the infinite horizon one can get the extensions of the above model. 
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moment t when the stop has been 



made by the y-th and the state of the 
renewal-reward process T/t , 5 

renewal-reward processes 

( r,,^ ' ' , X,j ' ' )-the renewal-rewarded 

processes, 6 
Fi{t)-the distribution function of the holding 

times of the /-th type, 4 
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7;,-n-th jump moment, 3 
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